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Abstract 

In view of space activities like International Space Station, Man-Tended-Free-Flyer 
(MTFF) and free flying platforms, the development of intelligent robotic systems is gaining 
increasing importance The range of applications that have to be performed by robotic 
systems in space includes e. g. the execution of expenments in space laboratories, the service 
and maintenance of satellites and flying platforms, the support of automatic production 
processes or the assembly of large network structures. Some of these tasks wul require t e 
development of bi-armed or of multiple robotic systems including functional redundancy. 

For the development of robotic systems which are able to perform this variety of tasks a 
hierarchically structured modular concept of automation is required. This concept is charac- 
terized by high flexibility as well as by automatic specialization to the particular sequence ot 
tasks that have to be performed. On the other hand it has to be designed such that the hu- 
man operator can influence or guide the system on different levels of control supervision, an 
decision This leads to requirements for the hardware and software concept which permit a 
range of application of the robotic systems from telemanipulation to autonomous operation. 
The realization of this goal requires strong efforts in the development of new methods, soft- 
ware and hardware concepts, and the integration into an automation concept. 


1. Introduction 

With respect to increasing space activities, e. g. ISS, free-flying platforms or planetary 
operations, it is necessary to reduce the operational costs for spare systems. One major key 
for future operational systems will therefore be the application of robotic systems in spare 
III. It is planned to use different kinds of manipulators and robots m spare to support and 
execute several tasks inside spare-modules or in free space, especially for 


- Docking/Berthing, 

- Repair and module exchange, 

- Service and maintenance ot free— flying platforms, 

- ORU (Orbit Replaceable Unit) - Exchange, 

- Assembly of large structures, 

- Experiment execution and production tasks. 


* This work was supported by a grant of the " Bundesminister fur Forschung und 
Technologie (BMFT)" ot Federal Republic Germany. 
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The execution of this variety of tasks for manipulators and robots in space reauires a 
hierarchically structured modular concept of automation including as well telemanimilation 
as autonomous operation. This design should cover the range of possible applications^ind has 
to provide interlaces for human interaction on different levels of control supervision nnH 
decision. To reach this goal, intensified efforts in the development of future-onented robotic 

incSKSteSi bS,iSg mU8rati0n iDt0 “ 0vera11 “"“P 1 of automation should be 

oJwered.T^ 

space is introduced. One of the key issues of the concept TlXnSfc Sk SSeSem 
which is diseased in section III. The structuring of the levels of coordinated operation and 
collision avmdance and a mathematical formulation of the substructures is presented in 
apter IV Finally a test facility for the proposed intelligent control structure and for soeci 
Whch was built up at I&F Laboratory in conn^ion ^fhThe natSl 
p oject CIROS (Control of Intelligent Robots in Spaoe), is described in section V. 


n. Overall System 

*.***? $** technol J °gy astronauts have to leave the space vehicle for almost everv 
execution of activities outside this vehicle. Robots and manipulators amid be^eTin space 

!® / educe tbe ns ^ and cost of such "Extra Vehicular Activities”, where robots and telemani- 
ric s^ / 2 / V StCP by SteP fr ° m Simple telemanipulation’robots fo C auto“mo“- 

mu n^^°? era k^° n * S * be brs t step in this development. Here a human operator controls the 
SySt ^ 1 ?^^ e ^i^ra^n°L\hown ii^fig^ 0 ™ Th^fim\evehsf ^^system^jTOentatbn'i^the 

S^ti* inC ' UdeS iD,ernaI Se ” SOrS tor the 

equJrions dynanUC m ° del ° f thC r ° b0t be described b y Wghly nonlinear differential 


i(t) = A00 + BOO u(t), x(t) = coo 


( 1 ) 


with nonlinear couplings between the variables of motion. In eq (1) xft) is the 
« S' ‘Xfvf ^ arC the m d ™ eosio " al '"Pit- and ompmiv£to£ 

f^ten^ntr^ order ' which “ the 


n(t) = £00 + GOO w(t) 
with 


( 2 ) 


£(g) = - d‘ l (Y) [ C*(x) + M*(X)] , SO) = - d' _1 (x) A 
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Figure 1. Control structure of a manipulator system for teleoperation 

where the matrices D*, C* and A are given from the nonlinear decoupling and control law 
/3/, one obtains a linear, decoupled behaviour for each axis: 

y s (t) + O.J y,(t) + a iQ yj (t) = Aj • w(t) , (■ = (3) 


The level of controllers computes signals for actuation which are transformed by the ac- 
tuator system into forces and torques acting on the mechanical construction /3/ . 

At this stage man is still in the loop, responsible for motion planning, coordination, 
collision avoidance and supervision. The operator may get help information from a know- 
ledge-based system, where system data can be stored and later on recalled for similar tasks. 
During teleoperation from ground time delays of up to 5 sec decrease the performance. 
Therefore it is necessary to provide the manipulator or robotic system even at this stage of 
development with a certain level of autonomy. 

Further steps towards autonomous robotic systems are telesupervision and teleauto- 
mation, where simple tasks are accomplished automatically, while the operator controls the 
system as telemanipulator for specific tasks. So the operator is not required to be perma- 
nently in the loop, which eases the job of the crew considerably. Due to the variety of the 
tasks, possible long duration missions and from aspects of reliability of the system the 
highest degree of automation is desirable. The last stage in this development is a fully auto- 
nomous operating multi-robot system, whose structure is given in fig. 2. The system consists 
of several robots with one or more arms each. Based on the structure of a manipulator 
system in fig. 1, the evolution to autonomous multi-robot systems is characterized by the 
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W lon of additional hierarchical levels. Strategies for automatic task management, coor- 
dinated operation and collision avoidance became integral parts of the structure. 
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Figure 2. Structure of an autonomous multi-robot system 


The levels for collision avoidance and coordinated operation are hierarchically placed 
above the level of coordinate transformation, which includes the various kinematics of the 
robots involved. The level of coordinated operation is responsible for the generation of refe- 
rcnce values, which enable a coordinated task execution. To avoid collisions of the robots 
with themselves as well as with obstacles, in the layer of collision avoidance appropriate 
strategies are implemented. The formulation of these strategies is based on a systematic 
design procedure for multi— robot systems / 4/, which has to be applied on a group and a 
system level. v 

The superimposed task management executive is responsible for automatic task activa- 
tion choice and reservation of appropriate robots, execution control and performance control. 
Also included are safety and emergency reactions, which are initiated in case of failure and 
contingency. The operation is assisted by a knowledge— based system, which r uns a model of 
environment as well as a task simulator, whereas the model of environment will be con- 
tinuously updated by evaluation of relevant sensor data and status reports. The task simula- 
tor checks out descriptions of new tasks, which are passed through the control supervision. 
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After a successful testing out, the resulting executable tasks will be transferred by the 
Meta-Controller to the sequencer for storage in a dedicated task memory, from which the 
program can be executed on demand. 

In this system, the task of the human operator reduces to initiation, supervision and 
acknowledgement of completed robot tasks by means of the control supervision unit. Never- 
theless, the operator is always able to take control over the system, especially in case ot 
failure or emergency situations. 


ITT Automatic Task Management 

In a large robot system, e. g. in a scenario of space station including robots with various 
capabilities, e. g. mobility, multi-armed systems, different working modes and on the other 
hand a broad range of very different tasks automatic task management is one of the key 
issues. Due to the complexity of the system itself and of the range of applications the pro- 
blem must be solved on an abstract level far beyond the level of move commands of commer- 
cial robot languages. Situated between the Meta-Controller as an interface to man and the 
system coordinator (CoS) as the intelligent interface to the single robots the automatic task 
management has to provide a lot of functions performing system control and the break down 
from the highest level of abstraction to a middle level at the robot side. At the input level 
complex tasks are described, which include implidtely the use of a group of appropriate 
robots for execution. The whole work of each task can be subdivided in parts which can be 
performed sequentially or in parallel according to the special needs of the task. 1 he system 
coordinator accepts coordination primitives, which address groups of robots on a multi ro- 
bot movement level. The break clown to collision free move commands for single robots is 
performed at the levels of coordinated operation and collision avoidance. The structure ot 
the levels mentioned is shown in principle in fig. 3 from the task input to the output to the 

robots. 


The tasks are transferred for execution from the Meta-Controller with a task specific 
global priority. The task management activates the task with the highest priority, it the 
capabilities ot the multi— robot system match the needs of the task. At this level groups of 
robots according to the task are defined but the robots are not yet booked Also the group 
coordinators (CoG) are configured and the group collision avoidances (CAG) are mitiallized. 


The choice and requisition as well as the derequisition of the robots with the appropriate 
performance capabilities is executed on the subtask level, where the subtasks provide a list ot 
performance attributes. The subtasks are initiated according to a set of rules, which consider 
priorities, the logic flow of execution, time critical paths and the availability of appropriate 
robot systems. The actual priority of a subtask is computed based on the global task prio- 
rity the attributes of the subtask and the current system status. For these reasons the 
system is event driven and the complete execution sequence is in general not known in ad- 
vance. Each subtask contains a number of coordination primitives, which are transferred to 
the system coordinator sequentially. These sequences are only broken m case of failure or 
emergency, while in ordinary operation the prescribed sequence of coordination primitives ot 
the subtask is executed. 


IV. Coordinated Operation and Collision Avoidance 

Due to the complexity of a space robotic system containing e.g. free-flying servicers, 
OMV' s RMS' s, SMS / s, etc. robot coordination and collision avoidance have to be con- 
sidered on two different levels. First on a global system level, which takes a global but rough 
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Figure 3. Substructure automatic task management, coordinated operation and collision 
avoidance 

view over all robot systems involved. Second on a group level, which takes a close look at a 
number of robots working at the same task or subtask. In order to bring out the highest 
degree of flexibility strong real-time capabilities are required on the group level. 

Considering a robot group insisting of r robots, each described by eq. (1) and controlled 
according to eq. (2), one obtains r sets of equations of form (3) for the closed loop robotic 
systems. As the robots work in coordinated operation, the reference inputs w^t), ..., w (t) 

have to be coordinated by a hierarchical coordinator of the type 

Wj(t) 

= Ac &1 *l *r)’ (4) 

w r (t) 

where Vj(t), ..., v^t) symbolize the move commands in coordination space for the nonlinear 
controlled robots. 

Applying the nonlinear control scheme eq. (2) to the individual robot arms it is /5/ 


146 







Aj • • • 0 x^t) 

• * * ♦ 

+ B • Hq (x v Xj.; v r v r ). (5) 

0 • • • A* x r (t) 

This formulation contains the dynamics A- (i = 1, r) of the axes of motion of all r 
robots and defines a second high level control loop by means of the hierarchical group coordi- 
nator Hq ( jc-^, •••» 2Lj.» ■••> Xj.)' 

The hierarchical group coordinator (CoG) eq. (5) is structured as follows eq. (6): 

x^t) ^(t) 

Bq ( x v •••> V .... Vy) = Hq • + (Xj, ■», x r ) + Eq ; 

x r (t) v r (t) 

- -J u 

Equation (6) is a formulation in joint space of the robots, which shows the applicability 
of this method. The dynamics of the links of the robots can be arbitrarily chosen by appro- 
priate selection of the matrix H a . Useful couplings between links of different robot can be 

introduced by H b . The matrix E contains gains for input vectors in coordination space /4/. 
The method of nonlinear decoupling offers a useful opportunity to choose the dynamics of 
the robotic system not only in joint space but in other e.g. task or group oriented coordinate 
systems as well. 




With an appropriate definition of the task variables the different modes of coordinated 
operation e.g. synchronisation, docking and cooperated payload handling can be implemen- 
ted, while tne structure of the coordinator remains the same /!/. Each group coordinator 
independently works on one coordination primitive and generates the input commands for 
the robots involved. The hierarchical system coordinator initializes the group coordinators 
allocates the coordination primitives and supervises the execution on a middle level, report- 
ing the system state to the units above. In case of a collision danger between single robots of 
different groups the system coordinator works close together with the system collision avoid- 
ance searching collision free but still group coordinated paths. This can be accomplished by 
priority considerations or a reconfiguration of the robot groups. 


The level of collision avoidance is responsible for a collision free operation of the whole 
system. This includes at first the realtime detection of collision danger between robots 
working in the very same group as well as between robots of different groups. Only in case of 
collision danger this module intervenes whereas in case of no danger of collision the original 
reference inputs are applied to the robots. The level of collision avoidance is split in two 
parts: collision avoidance on group level (CAG) i.e. between robots of the very same group 
and on system level (CAS) i.e. between robots of different groups. The group collision avoid- 
ance has to distinguish three major operation modes: independent action, synchronised 
actions and fully cooperated motion of the robots involved. It is obvious that the avoidance 
strategy is closely dependent on the mode of operation, but in each case the strategy can be 
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described by a similar structure as the group coordinator eq. (6): 



( 7 ) 


In this formulation the states and inputs of all robots have been noted in a condensed form 




vf =(Y 


T 


*rl> 


( 8 ) 


For the collision avoidance between robots of the very same group (xj) in eq. (7) is the 

essential part of the structure. The detection of danger of collision as well as the collision 
avoidance strategy are based on the calculation of the collision avoidance trajectories f., 

(j = 1> •••> 0- These trajectories are determined by on-line prediction of the robot 
movement, regarding the current robot states the preprogrammed paths and the task 
oriented right-of-way priorities of the robots /6/. These trajectories are described in the 

elements of the matrix including the information if currently collision danger was 

detected. The matrices permits the change of the control dynamics in dependence on the 
level of danger of collision and the original inputs Vj are cancelled by means of Eq in case of 
a predicted collision. 

Considering now a multirobot system consisting of N groups of r^ (i = 1, ..., N) robots each, 

where all robots are feedback controlled according to eq. (2) and each group equipped with a 
CAG-unit. This formulation leads to N robot groups, which are completely decoupled where 
the single robots of the very same group are in coordinated operation by means of CoG and 
under online collision avoidance by means of CAG. The case of collision avoidance between 
robots of different groups is implicitely included by the demand for collision free paths v- of 

robots of different groups, as inputs to CAG from the CAS unit 

The separation of collision avoidance on group and system level, respectively takes use of 
the fact, that mostly collision danger occurs inside a robot group, where the robots work 
close together. Here the group dedicated CAG provides a fast response in case of detected 
collision danger inside the group, taking the working mode of the robots into consideration. 
In case of collision danger between robots of different groups, which occurs less frequently 
e.g. with mobile robot systems but is of the same importance with respect to possible 
damage of the systems, the CAS intervenes. It generates a vector e containing the draw back 
directions, which decrease the danger of collision the most. Based on this information the 
CoS and CoG react, where the possible constraints of coordinated operation are still kept. If 
conflicts or deadlock situations occur the strongest action is the reconfiguration of the groups 
and the CoG s taking the conflicting systems in the very same group. Otherwise evasive 
actions and priority increase of the conflicting robots inside the group can solve the problem. 


V. CIROS test facility 

To provide a realistic environment for development and test of the modules of the pro- 
posed hierarchical control structure for robots in space an appropriate facility was built up 
in the IRF laboratory, which is part of a national space project called CIROS (Control of 
Intelligent Robots in Space). Based on the modular concept it is possible to develop and 
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implement methods and strategies based on terrestrial robots, because for the transfer to a 
real space environment only the robots themselves and the low level control up to the coordi- 
nates transformation have to be changed. The upper levels of the control structure however 
remain the same with some minor adaptions. As test scenario an unmanned space laboratory 
(e g Spacelab, MTFF) was chosen, where typically experiment service repair tasks or expe- 
riment exchange are to be performed. The test facility is completed by a control and super- 
vision board which could be integrated in a manned space station or in a ground based con- 


trol center. 


ORIGINAL PAGE IS 
Of POOR QUALITY 


Figure 4. 



Robots with common working space in the CIROS test facility 


This environment provides the test facilities for all the upper levels of the control 
structure including different grades of automation. In order to study the problems in 
multi-robot systems for space applications two robots with widely overlapping working 
spaces are integrated. Both robots are equipped with tool exchange capabilities and additio- 
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nal sensors e.g. force— torque sensor, arm— mounted camera, proximity sensors ect. So each 
robot is able to perform every task in the system. Therefore it is possible to consider the 
automatic task management as well as coordinated operation and collision avoidance in a 
realistic environment. The hierarchical control is implemented on a real-time computer, 
which is interconnected with a knowledge based system and the control supervision, where 
time delay can be simulated between the different computers. The control board contains 
several input/output devices like alpha numeric terminals, graphics, video sensor ball, ect. 
From this board the system is supervised, a runtime documentation is done and inter- 
ventions of the human operator are accepted. Additionally it is used as development facility 
and an off-line programming system is integrated as well as a cell simulation. A picture of 
the two used robots working on a rack is shown in fig. 4. In this cell, which was designed 
similar to a spacelab environment, the main functions of the hierarchical control structure 
can be implemented and demonstrated exemplarilv. Especially the problems and capabilities 
of multi-robot systems for space applications can be studied. 


VI. Conclusion 

In this paper a hierarchical structure for the control of multi robot systems for space 
applications is presented. The break down from a high level of abstraction at task manage- 
ment level down to the single robot control is described step by step. The splitting in a con- 
sideration on system and on group level takes the distributed character of a large space 
system into account. As a possible space scenario for A&R an unmanned space station is 
focussed and introduced as development and test environment at the IRF— Laboratory. 
Based on this facility A&R with multi— robot systems can be studied at IRF in practical 
examples. 
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